Memory devices attachable to data processing units (hosts), such as central processing units, multiprocessors and networks of such processors, either directly or via diverse communication links, are often arranged in hierarchies in accordance with performance and other parameters. Such devices are often controlled through computer programs that maintain active data sets on fast-access storage devices, while less active data sets are maintained on lower-cost storage devices. As an example, in an installation employing International Business Machines Corporation (IBM) peripheral devices, active data sets are often kept on a model 3380 disk storage apparatus while less active data sets can be stored in an IBM 3850 Mass Storage System. Further, the disk storage apparatus can be divided into a hierarchy based upon cost. Again, the IBM 3380 would be in a primary set of storage devices for operating directly with the host data processing system. Older disk storage apparatus, such as the IBM 3330 disk storage apparatus, can be in a secondary disk storage apparatus and serve as secondary volumes for storing somewhat less active data sets than those stored on the IBM 3380 disk storage apparatus.
Utilization of memory devices always employs allocation and deallocation techniques. That is, the addressable registers of the various memory devices, when allocated, are designated as being usable only by designated hosts of the data processing system or processes within a host, or for a collection of processes or hosts. Allocation is merely taking an unassigned register, herein termed as free space, and designating it for a process or a host.
For example, in so-called main memory, i.e., the memory which a host uses as a main operational memory, empty registers are identified in a queue of registers such that allocation can occur quickly. Tomasulo et al., in U.S. Pat. No. 3,241,125, employ such a technique. This technique is suitable for main memory applications; however, when a large plurality of peripheral storage devices, such as disk storage apparatus and the like, are employed, because of the large storage capacities found in peripheral storage devices, this method becomes cumbersome. As used herein, the term peripheral storage device is intended to mean a storage device having a relatively large storage capacity; such storage device can be used at the periphery of the host or can be integral thereto; this invention can apply to either application.
Generally, newer technology storage devices, such as the IBM 3380 compared to the IBM 3330 storage device, tend to be larger capacity and faster access. The amount of data storable in a given device is often referred to as a "volume of data". The IBM 3850, for example, employs addressing based upon the storage capacity of the IBM 3330 device and termed as "virtual volumes". Accordingly, the performance of a storage device can be related to the capacity of that device. This observation is particularly pertinent to operating large-capacity storage devices in that so-called data-streaming techniques can be employed for transferring large amounts of data between storage devices, between host and storage devices, through data communication links, and the like. Generally, efficiency of the entire system is improved when data streaming is employed. Accordingly, it is desired to create a greatest probability to successfully complete data-streaming transfers to a large number of storage devices. It is also desired to manage the space of all such storage devices in an extremely low-cost and efficient manner.
Most storage devices employ so-called directories or volume table of contents (VTOC) which indicate the data stored in the memory device or volume, the amount of free space, and other control information interesting for the management of such devices. This technique is also employed in main memory such as shown in U.S. Pat. No. 4,047,160, which has a bit indicating an unallocated or free register. While a separate indicator for each unit of free space may be efficient and fast for main memory applications, scanning the VTOC's of a plurality of large capacity storage devices is quite time consuming; hence, inefficient. Accordingly, it is necessary to provide a new control mechanism for managing space on this type of storage device.
It is also desired to maximize efficiency of storage devices based upon balancing the loads between the various storage devices. This is particularly important not only in plural main memories but also in peripheral input/output type of storage devices, commonly referred to as peripheral devices. U.S. Pat. No. 3,702,006 shows a load-balancing technique for peripheral devices. This particular technique is not related to the inventive concept to be described in this document; however, it does show a way of balancing peripheral device operations. It is now desired to provide load balancing peculiarly adaptable to data streaming, i.e., large data transfers between a plurality of devices with techniques for ensuring the successful completion of such extended data transfers.
It would certainly be advantageous if a simple control mechanism and method could achieve, in many applications involving data streaming, all of the above desired goals. An example of such data streaming control is the migration of data sets between peripheral storage apparatus volumes, such as from a 3380 disk apparatus to a 3850 Mass Storage System, when a data set becomes less active.